gh-143756: Fix potential data race in SSLContext.load_cert_chain#143818
Conversation
Concurrent calls to `load_cert_chain` caused data races in OpenSSL code.
picnixz
left a comment
picnixz
left a comment
There was a problem hiding this comment.
Do we need tests for that actually?
| do { (save) = PyEval_SaveThread(); } while(0) | ||
| #define PySSL_END_ALLOW_THREADS_S(save) \ | ||
| do { PyEval_RestoreThread(save); _PySSL_FIX_ERRNO; } while(0) | ||
| #define PySSL_BEGIN_ALLOW_THREADS(self) { \ |
There was a problem hiding this comment.
While you are it, can you use a do-while construct? (here we would just have the do {) In addition can you make it multiline with line continuations aligned on a tab multiple? (like PEP-7) TiA.
| do { (save) = PyEval_SaveThread(); PyMutex_Lock(mutex); } while(0) | ||
| #define PySSL_END_ALLOW_THREADS_S(save, mutex) \ | ||
| do { PyMutex_Unlock(mutex); PyEval_RestoreThread(save); _PySSL_FIX_ERRNO; } while(0) | ||
| #define PySSL_BEGIN_ALLOW_THREADS_S(save) \ |
There was a problem hiding this comment.
Can you use uppercase letters for the macro parameters please?
There was a problem hiding this comment.
Let's not do a bunch of extra refactoring in a bugfix PR
There was a problem hiding this comment.
Oh right you want to backport this. I think there will be anyway conflicts so yeah let's not make it more complicate
| if (keyfile == Py_None) | ||
| keyfile = NULL; |
There was a problem hiding this comment.
I think this can be added in the if (keyfile) check instead.
There was a problem hiding this comment.
ditto; this is existing code
| if (password != Py_None) { | ||
| if (PyCallable_Check(password)) { | ||
| pw_info.callable = password; | ||
| } else if (!_pwinfo_set(&pw_info, password, |
There was a problem hiding this comment.
| } else if (!_pwinfo_set(&pw_info, password, | |
| } | |
| else if (!_pwinfo_set(&pw_info, password, |
| if (PyCallable_Check(password)) { | ||
| pw_info.callable = password; | ||
| } else if (!_pwinfo_set(&pw_info, password, | ||
| "password should be a string or callable")) { |
There was a problem hiding this comment.
And realign this adter putting the elseif on its own line
| if (!PyUnicode_FSConverter(certfile, &certfile_bytes)) { | ||
| if (PyErr_ExceptionMatches(PyExc_TypeError)) { | ||
| PyErr_SetString(PyExc_TypeError, | ||
| "certfile should be a valid filesystem path"); |
There was a problem hiding this comment.
For a follow-up: This is a bit misleading. For me such message usually implies a ValueError instead of TypeError. Do we use this formulation for other occurrences of FSConverter?
Maybe: "expecting a path-like object for 'certfile', got ..."
|
Tests already exist. #143752 runs it with TSan |
picnixz
left a comment
picnixz
left a comment
There was a problem hiding this comment.
Sorry as I was on mobile I didn't see that it was just moving code around. We can refactor this later (I really want to split ssl.c because it is messy to maintain but it is also annoying to create more files...)
pythongh-143818) Concurrent calls to `load_cert_chain` caused data races in OpenSSL code.
Concurrent calls to SSLContext.load_cert_chain on the same context race on the SSL_CTX default password callback. This is fixed in 3.15 (pythonGH-143818), but under TSAN with instrumented OpenSSL the race is now reported on 3.14, so skip the test that triggers it.
This test spawns 8 threads calling load_cert_chain on the same context, directly triggering the data race on the SSL_CTX default password callback (fixed in 3.15 by pythonGH-143818). It was the test that failed both TSan CI jobs.
…53355) Compile OpenSSL with TSan in the TSan CI job so that data races between Python code and OpenSSL internals are visible to the sanitizer. Also skip test_ssl_in_multiple_threads under TSan: concurrent calls to SSLContext.load_cert_chain on the same context race on the SSL_CTX default password callback. The race is fixed in 3.15+ (GH-143818), but with TSan-instrumented OpenSSL it is now reported on 3.14. (cherry picked from commit fc19ad7) Co-authored-by: Sam Gross <colesbury@gmail.com>
2 participants
Concurrent calls to
load_cert_chaincaused data races in OpenSSL code.